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This report has been prepared under Section 6 of the National Audit Act, 
1983 for presentation to the House of Commons in accordance with Section 


9 of the Act. 
John Bourn National Audit Office 
Comptroller and Auditor General 19 November 1991 


The Comptroller and Auditor General is the head of the National Audit 
Office employing some 900 staff. He, and the NAO are totally independent of 
Government. He certifies the accounts of all Government departments and a 
wide range of other public sector bodies; and he has statutory authority to 
report to Parliament on the economy, efficiency and effectiveness with 
which departments and other bodies use their resources. 
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Summary and conclusions 


Origin and scope 
of the examination 


1 


4 


Published sources suggest that total Government expenditure on research 
and development in 1989-90 amounted to some £4.8 billion, of which about 
£2.2 billion related to defence and over £2.6 billion to the civil sector. Of the 
£2.2 billion defence total, the Ministry of Defence (the Department) have 
reported that slightly over two fifths of this expenditure is incurred 
intramurally mainly at their research establishments and that the remaining 
three fifths relate to commissioned work mainly from private industry 
(paragraphs 1.11 and 1.15). 


Statistics on research and development expenditure are used by 
Government as a management tool to help control expenditure, to guide 
policy and as an indicator of national investment in technological and 
scientific expertise. The definition of research and development adopted 
Government-wide in the Statistical Supplement to the Chancellor’s Autumn 
Statement and in an Annual Review issued by the Cabinet Office is that 
contained in the “Frascati Manual” prepared by the Organisation for 
Economic Co-operation and Development. The Central Statistical Office 
compile the statistics in the Annual Review and are responsible for ensuring 
that they are sound and on a consistent basis, both domestically and 
internationally (paragraphs 1.1 to 1.6). 


According to the Frascati Manual the basic criterion for distinguishing 
research and development from related activities is the presence in research 
and development of an appreciable element of novelty —a principle 
accepted as authoritative by the twenty four member countries of the 
Organisation for Economic Co-operation and Development. The Frascati 
Manual is also the foundation for the United Kingdom Statement of 
Standard Accounting Practice 13 which, since its revision in January 1989, 
provides for the disclosure of research and development expenditure by 
public limited companies as good accounting practice (paragraph 1.4). 


In their 3rd Report of 1989-90 on Definitions of Research and Development, 
the House of Lords Select Committee on Science and Technology concluded 
that much of what was classed as defence research and development was 
not true research and development at all. The Committee recommended that 
the National Audit Office, with suitable technical support, should report on 
the Department’s research and development expenditure, having identified 
how much of the expenditure fell within the Frascati definitions and 
Accounting Standard 13 (paragraph 1.16). 


In early 1990, the Department embarked upon a major review of the 
application of the Frascati definitions to their research and development 
expenditure. The National Audit Office examination seeks to add to the 
debate in a constructive way but does not attempt to duplicate the 
Department’s investigation. 


In undertaking this examination, the National Audit Office recognised that 
there were value for money issues involved in providing accurate 
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On Frascati and 
related definitions 
of research and 
development 
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information for Ministerial decision-making and to Parliament. The report 
examined: 


(i) the applicability of the Frascati definitions and other related definitions 
of research and development, in particular their relevance to defence 
(Part 2}: 


(ii) the compilation of research and development expenditure statistics 
and the problems encountered in applying the Frascati definitions in 
the defence and civil sectors of the United Kingdom and overseas (Part 
3); 

(iii) the measures taken by the Department to implement the Frascati 
definitions and the effects of full implementation both on the 
Department and on Government statistics as a whole (Part 4). 


The National Audit Office’s findings and conclusions are set out below. 


In 1993 the Organisation for Economic Co-operation and Development 
expect to publish a revised version of the Frascati Manual. Despite some 
shortcomings in both the Frascati definitions and Accounting Standard 13 
(revised), the National Audit Office agree with the general view of the 
accountancy profession and industry that the Frascati/accounting standard 
framework provides a means of achieving some measure of consistency in 
the provision of statistics for research and development expenditure. The 
National Audit Office also agree with the Department that the Frascati 
Manual needs to provide more specific guidance in relation to defence to 
ease interpretation (paragraph 2.2 to 2.4). 


The House of Lords Select Committee in their recent report have made 
suggestions for improving the Manual which include making the Manual 
more user friendly; considering the benefits accruing from the application of 
Accounting Standard 13; and separating civil from defence research and 
development. In the Manual, research and development are measured in the 
form of inputs: expenditure on research and development, and people 
working in research and development. But the Confederation of British 
Industry have pointed out that inputs can only be a proxy measure of 
competitive success in the absence of the measurement of research and 
development outputs (paragraphs 2.5 to 2.7). 


Harmonisation of accounting standards on a worldwide basis is the current 
trend but to date there has been no dialogue between the Organisation for 
Economic Co-operation and Development and the accountancy profession. In 
January 1991, the first report to be sponsored by the Board for Chartered 
Accountants in Business concluded that the revised Accounting Standard 13 
is limited in scope and provides inadequate guidelines to ensure consistent, 
comparable measurement of research and development expenditure. The 
National Audit Office note that the Accounting Standards Board are 
currently reviewing experience of the revised standard (paragraphs 2.14 and 
a7). 


In the National Audit Office’s view the Central Statistical Office should 
encourage the Organisation for Economic Co-operation and Development to: 


e introduce simplified defence guidelines into the revised version of the 
Frascati Manual; 
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e consider incorporating into the revised Manual the various suggestions 
for improvement made by the House of Lords Select Committee on 
Science and Technology; 


e give priority to development comprehensive research and development 
output measures; 


e enter into a dialogue with the accountancy profession to ensure that 
research and development accounting standards worldwide are 
compatible with the Frascati definitions. 


12 No basic curiosity-driven research as defined by the Organisation for 
Economic Co-operation and Development is undertaken by the Department. 
The development work of the Department is related to specific equipment 
projects and is managed in three consecutive but distinct stages: feasibility 
study, project definition and full-scale development. The extent to which 
these three phases fall within the Frascati definition of experimental 
development is not clear but all three phases probably include some 
elements of Frascati and non-Frascati work (paragraphs 2.9 and 2.11). 


13 The Department therefore should: 


e analyse the extent to which each of their three phases of development 
falls within the Frascati definitions of experimental development. 


On the compilation 14 Nearly all member countries of the Organisation for Economic Co-operation 


of research and and Development have some problems in applying the Frascati definitions; 
development the United Kingdom is not unique in this respect. Common problems are the 
expenditure exclusion of small firms in industry surveys and discrepancies between 
statistics and Government and industry reporting. The Organisation compile international 
associated problems statistics mainly from their own Biennial Benchmark International Survey. 


The United Kingdom undertakes a quadrennial survey of industry (or 
census) updated annually by sample surveys whereas most other major 
nations undertake an annual census or at least a biennial one (paragraphs 
3.2 to 3.5). 


15 The National Audit Office take the view that the Central Statistical Office: 


e should consider undertaking either an annual or biennial survey (or 
census) of industry in line with most other nations. 


16 The Department, as the funder, must report on intramural and extramural 
research and development expenditure for the calculation of the United 
Kingdom’s total Government funded research and development. Also, the 
Department, as the performer, must report on intramural expenditure alone 
for the calculation of the United Kingdom’s gross expenditure on research 
and development. The figures reported by the Department are taken from 
the vote accounting system but this information is not sufficiently 
disaggregated to allow further analysis of Frascati and non-Frascati 
elements. The bulk of any potential misclassification is likely to occur at the 
development/production interface, a definitional ‘grey’ area (paragraphs 3.12 
and 3.13). 


17 On intramural research and development, the Department have concluded 
that generally sufficient information is held at their research establishments 
to enable estimates of expenditure falling within the Frascati definitions to 


a a ee 
3 


CLASSIFICATION OF DEFENCE RESEARCH AND DEVELOPMENT EXPENDITURE 


18 


19 


be improved. Extramural research and development is the area where the 
Department experience most difficulty in identifying expenditure meeting 
the Frascati definitions. Detailed information would be needed of how 
contract payments are spent but the Department do not have this 
information and neither may defence contractors, particularly where work is 
sub-contracted. Some defence companies may be reporting research and 
development to accord with the Frascati definitions but others may be using 
the Department’s classification of procurement (paragraphs 3.14 and 3.17). 


In 1989-90 the United Kingdom Government reports having paid £333 
million more to industry than industry claims to have incurred on 
Government work. This discrepancy has reduced from about £500 million in 
1985-86. For defence the discrepancy has reduced from £235 million to £213 
million between 1985-86 and 1989-90. The defence discrepancy is currently 
being investigated by a working group comprising the Central Statistical 
Office, the Department and the defence industry (paragraphs 3.19 to 3.22). 


The National Audit Office believe that the Department, in liaison with the 
Central Statistical Office and defence industry, should: 


e concentrate their efforts on resolving misclassification in the ‘grey’ area 
between development and production; 


e consider undertaking a sample survey of defence contractors to establish 
the extent to which they can supply Frascati data on the Department’s 
extramural activities; 


e investigate the extent to which companies are not reporting research and 
development to accord with the Frascati definitions; 


e give priority to resolving the discrepancy between the sum the 
Government reports as having paid to industry and what industry reports 
as having incurred on Government funded work. 


On the action taken 
to implement the 
Frascati definitions 
and the implications 
of full 


implementation 
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Many wider benefits, such as uniformity of reporting and more informed 
decision-making, might accrue if the Department were to improve the 
alignment of their figures with the Frascati definitions. By means of 
statistical surveys, the Department have already gone to considerable 
lengths to identify the potential for misattribution of research and 
development expenditure in the published estimates. The Department 
undertook a survey in 1987 of a sample of projects. For this sample, about 79 
per cent of the reported research and development expenditure appeared to 
fall within the Frascati definitions. This showed a potential for 
misattribution of expenditure to research and development. Although this 
result must be treated with caution, if it were typical it would suggest that 
for 1989-90 Government funded defence research and development 
amounted to about £0.5 billion less than the reported £2.2 billion. Currently 
the Department are taking positive action to try to establish whether, and if 
so why, any over or under reporting is occurring, eg the working group with 
the Central Statistical Office and industry (paragraphs 4.2 to 4.6, 4.11 to 
4.23). 


The Department believe that there is no domestic benefit to them in fully 
implementing the Frascati definitions, and see difficulties in incorporating 
such a break-down into the vote accounting structure taking into account 
the size, complexity and cost of the exercise. Similarly, the introduction of a 
contract condition to obtain the necessary Frascati information on 
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extramural research and development from industry would be very 
problematic. The National Audit Office note the necessity of ensuring that 
Frascati information on intramural research and development can continue 
to be made available from the research establishments of the new Defence 
Research Agency (paragraphs 4.27 to 4.33). 


Before taking further action in their attempts to comply more fully with the 
Frascati definitions the Department should, in the National Audit Office’s 
view: 


e come to an agreement with the Defence Research Agency to obtain the 
necessary Frascati information; 


e investigate the extent to which the defence industry is able to supply 
Frascati information on extramural research and development 
expenditure and is willing to provide the information on a voluntary 
basis; 


e analyse the cost implications and benefits carefully before requiring 
defence contractors to supply information which would enable 
expenditure meeting the Frascati definitions to be identified. 


Since the mid 1980s, there has been a growing recognition within the 
Department that the figures they were reporting in the Cabinet Office 
Annual Review might be capable of improved alignment with the Frascati 
definitions. As Part 2 of this Report shows, there is a need to revise and 
simplify the Frascati definitions and also to ensure that accounting standards 
fully comply with them. Accounting standards will have to be brought in 
line with the revised Frascati Manual due to be published in 1993. But 
taking the definitions as they stand at present the Department have not been 
able to identify with complete assurance which elements of their research 
and development expenditure are truly within the Frascati definitions. They 
need to investigate this further. 


There are also doubts about the extent to which the defence industry is 
reporting expenditure to accord with the Frascati definitions. The 
Department, together with the Central Statistical Office and the defence 
industry, are trying to resolve the discrepancy between the figures on 
extramural research and development reported by the Department and 
industry. The National Audit Office take the view that sustained action by 
the Department is required on this issue. 


If, at the conclusion of the current exercise, the Department find that they 
are unable to produce an accurate Frascati estimate for extramural research 
and development, they will need to consider whether there is an overall 
advantage in imposing new reporting requirements on industry by 
introducing a special contract condition. This could have repercussions on 
the Department’s present procurement policy geared as it is to competitive 
tendering and a “hands off” approach to industry. Also, the long-term 
prospect of incorporating Frascati into the Supply Estimate and the vote 
accounting structure is likely to have serious consequences for the 
Department in terms of cost and disruption. There is no easy solution here. 
Before embarking on major changes to their procurement contracts and to 
the vote structure, the National Audit Office urge the Department to 
establish the cost of these innovations and to weigh this up carefully against 
the reporting requirements and the likely wider benefits. 
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Part 1: Introduction 


The purpose and reporting of 
research and development 
expenditure 


Statistics on research and development 
expenditure are used by Government as a 
management tool to: 


e determine overall research and 
development policy and priorities; 


@ manage and control research and 
development expenditure inter- 
departmentally; 


e monitor the adequacy of research and 
development investment in different 
sectors of the economy; and 


e make comparisons of research and 
development internationally. 


Expenditure statistics on research and 
development for all Government departments 
are reported annually in the Statistical 
Supplement to the Chancellor’s Autumn 
Statement, in the Cabinet Office’s Annual 
Review of Government Funded Research and 
Development; and also for defence in the 
Statement on the Defence Estimates. 
Moreover, the United Kingdom has an 
obligation to provide annual details of 
Government research and development 
expenditure to the European Community and 
to the Organisation for Economic Co- 
operation and Development. 


Definition of research and 
development 


The definition of research and development 
adopted Government-wide in the Statistical 
Supplement to the Chancellor’s Autumn 
Statement and in the Cabinet Office’s Annual 
Review is that contained in the ‘Frascati 
Manual” prepared by the Organisation for 
Economic Co-operation and Development. 
Standard research and development 
definitions were first agreed at the 
Organisation’s 1963 conference at Frascati in 
Italy and thereafter have been termed the 
Frascati definitions after the venue. 


1.4 


1.5 


1.6 


According to the Frascati Manual, the basic 
criterion for distinguishing research and 
development from related activities is the 
presence in research and development of an 
appreciable element of novelty. This 
principle is accepted as authoritative by the 
Organisation’s twenty four member countries 
which include the United Kingdom, the 
United States, France, Italy, Japan and the 
former Federal Republic of Germany. The 
Frascati Manual is also the foundation for the 
United Kingdom’s Statement of Standard 
Accounting Practice 13, which, since its 
revision in January 1989, provides for the 
disclosure of research and development by 
public limited companies as good accounting 
practice. The Appendix provides further 
details of the Frascati definitions. 


Roles and responsibilities 


At the international level, the Organisation 
for Economic Co-operation and Development 
are responsible for compiling statistics on 
research and development expenditure in the 
main industrial nations. The Central 
Statistical Office co-ordinate the submission 
of the United Kingdom’s statistics and 
represent the United Kingdom in any 
discussions with the Organisation on the 
application of the Frascati definitions. 


From the data provided by Government 
departments and industry, the Central 
Statistical Office compile the United 
Kingdom’s statistics which are published in 
the Cabinet Office’s Annual Review of 
Government Funded Research and 
Development. For the compilation of the 
United Kingdom’s gross expenditure on 
research and development, Government 
departments and industry report on the 
expenditure incurred in performing research 
and development in house — this is in 
accordance with the Frascati Manual. For 
compilation of the nation’s Government 
funded research and development, 
Government departments report on sums 
spent intramurally at their own research 
establishments etc. and extramurally on 
commissioned work. The Central Statistical 
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Office are responsible for ensuring that these 
figures are statistically sound and on a 
consistent basis both domestically and 
internationally. The Ministry of Defence (the 
Department) and the defence industry are 
required to produce accurate statistics which 
relate to defence. 


International comparisons 


Of the twenty four member countries in the 
Organisation for Economic Co-operation and 
Development, there are seven with major 
research and development expenditure: the 
United States, Japan, France, the United 
Kingdom, Italy, West Germany and Canada. 
Sweden also spends a substantial sum on 
research and development, a good proportion 
of which is on defence. For the purpose of 
making international comparisons, national 
gross expenditure on research and 
development as a percentage of gross 
domestic product is often the basis used. As a 
percentage of gross domestic product, the 
United Kingdom’s total research and 
development expenditure from all sources, 
both public and private, is the same as 
France but is much less than the United 
States, Sweden, West Germany and Japan 


1.8 


TAS) 


with only Italy and Canada spending less 
(Figure 1). 


On Government funded defence research and 
development, in terms of gross domestic 
product, only France and the United States 
spend more than the United Kingdom. 
However, for Government funded civil 
research and development in relation to gross 
domestic product, the United Kingdom ranks 
slightly higher than Canada, Japan, and the 
United States but below the rest (Figure 2). 


In total the members of the Organisation for 
Economic Co-operation and Development 
spend about 40 per cent of their 
Governments’ research and development 
funds on defence programmes. There is a 
marked concentration of defence research 
and development in a few countries, the 
United States alone being responsible for 
about 80 per cent of the total; and the top 
three defence spending Governments, the 
nuclear powers of the United States, the 
United Kingdom and France, are responsible 
for nearly 95 per cent. The United States 
spends about two thirds of its Government 
research and development budget on 
defence, whereas the United Kingdom spends 
about a half and France over a third. Other 


Figure 1: National gross expenditure on R&D for 1989 as a percentage of gross domestic product (GDP) for 


the United Kingdom and 7 other major OECD countries — performer based data 


Percentage of GDP 
3.0 


2a 
2.4 


2.1 


0.0 


Canada Italy UK 
Source: OECD Main Science and Technology Indicators 


France 


Sweden Japan 


Figure 1 shows that the United Kingdom’s gross research and development expenditure from all sources as a percentage of gross domestic 
product is similar to France but less than the United States, Sweden, West Germany and Japan. 
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Figure 2:'Government funded R&D for 1989 as a percentage of gross domestic product (GDP) for the United 
Kingdom and 7 other major OECD countries — funder based data 
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Figure 2 shows that on Government funded defence research and development, in terms of gross domestic product, only France and the United 
States spend more than the United Kingdom. 


nations which spend a sizeable portion of and private, was £11.5 billion for 1989. Table 
their Government research and development 1 below shows the source of funds and the 
budget on defence are Sweden (a quarter), performers in a simplified form. This table is 
West Germany (an eighth) and Italy (a tenth). compiled by the Central Statistical Office 


from data on intramural research and 
development expenditure provided by the 


The United Kingdom’s gross performers ie Government, industry and 
expenditure on research and others. For example, Government 
development departments have reported that out of a total 


of £3,449 million performed in house on 
research and development (more than 

half this figure relates to higher education), 
£2,830 million was funded by Government, 


Published sources suggest that the United 
Kingdom’s total investment in research and 
development from all sources, both public 


Table 1: The United Kingdom’s gross expenditure on research and development in 1989 as reported by the 
performers 


Performer 
Government Industry Other Total 
(i) (ii) Non-Government Financed 
Source of funds £m £m £m £m 
Government (i) 2,830 1,249 216 4,295 
Industry (ii) 289 5,328 192 5,809 
Other (including overseas) 330 1,023 75 1,428 
Total Performed 3,449 7,600 483 11,532 


Source: Cabinet Office Annual Review of Government Funded Research and Development 1991, Table 1.4.2. 
Notes: (i) Government includes higher education. 
(ii) Industry includes public corporations and research associations. 


Table 1 shows that the United Kingdom's gross expenditure on research and development in 1989 was £11.5 billion and analyses this total in 
relation to the main sources of funding and performers. 
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£289 million by industry and £330 million 
from other sources. The figures in Table 1 are 
provided to the Organisation for Economic 
Co-operation and Development as the basis 
for making international comparisons. 


Defence in the context of total 
United Kingdom Government 
expenditure on research and 
development 


Published sources also suggest that the total 
United Kingdom Government expenditure on 
research and development in 1989-90 was 
some £4.8 billion, of which about £2.2 billion 
related to defence and over £2.6 billion to the 
civil sector (Figure 3). These statistics are 
compiled from the annual figures of 


Ae lZ, 


intramural and extramural expenditure 
reported by Government departments. The 
difference between the £4.8 billion 
expenditure in 1989-90 reported by 
Government in Figure 3 and the £4.3 billion 
in 1989 reported by performers in Table 1 is 
due mainly to a discrepancy of £333 million 
between the £1,582 million reported by 
Government as having been paid to industry 
and the £1,249 million reported by industry 
as having been incurred on Government 
work (Table 1 and paragraph 3.19). 


The research and development programme of 
the Department has the objective of meeting 
the needs of the Armed Services for 
equipment and weapons in a timely and cost 
effective manner. The research programme is 
aimed at producing an underlying basis of 


Figure 3: United Kingdom Government research and development expenditure 1987-88 to 1991-92 


£ million (real terms, base year 1989-90) 
6000 


5000 


4000 


3000 


2000 


1000 


1988-89 


1987-88 


1987-88 


Bie) Civil R&D 


2747.1 


4 Total R&D 


5049.6 


1989-90 


1988-89 % 1989-90 % 
54,4 2672.3 
[_] petence Rap | 23025 45.6 21183 44.2 2160.7 44.7/2148.5 44.9 1951.4 44.3 


4790.6 


55.8 2670.5 


1991-92 


1990-91 


Estimate Provision Plans 


1990-91 % 1991-92 % 


55.3 | 2633.5 55.1 2455.9 55.7 


4831.2 4782.0 4407.3 


Source: Cabinet Office Annual Review of Government Funded Research and Development 1991, Tables 1.2.2 and 1.2.3. 


Note: Expenditure figures are VAT exclusive. 
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Figure 3 shows that the defence share of total Government research and development fell from 45.6 per cent in 1987-88 to range between 44.2 


and 44.9 per cent during the years 1988-89 to 1991-92 whereas the civil share rose from 54.4 per cent in 1987-88 to fluctuate between 55.1 per 


cent and 55.8 per cent during the years 1988-89 to 1991-92. 


CLASSIFICATION OF DEFENCE RESEARCH AND DEVELOPMENT EXPENDITURE 


10 


scientific and technological expertise for 
application in the selection, development, 
production and operation of weapon systems 
and equipments. Development work, 
although drawing on the knowledge and 
expertise obtained from research is directly 
related, item by item, to the procurement of 
specific military equipments, such as 
particular aircraft or radars, and is the 
essential forerunner to the production of 
those equipments. 


Since 1987-88, the United Kingdom 
Government’s policy has been that defence 
should not absorb an excessive share of the 
nation’s technological resources; the 
Department are therefore aiming to achieve a 
gradual and continuing reduction in the real 
level of research and development 
expenditure over the next decade. The 
Department’s intention is to maximise 
international collaboration and off-the-shelf 
procurement where this brings value for 
money. 


tamks) 


In real terms over the five years 1987-88 to 
1991-92, Government expenditure on all 

research and development reduced by 12.7 
per cent from £5,050 million in 1987-88 to 


Figure 4: Sectors performing United Kingdom Government funded defence research and development 


Ministry of Defence 
41% 


Private Industry 
51% 


Source: Cabinet Office Annual Review 1991, Table 2.2.2. 
Notes: 


(i) Percentages are based on gross defence research and development expenditure for 1989-90. 
(il) Private industry 51 per cent includes public corporations and research associations. 
(iii) Other 8 per cent includes civil Government departments, research councils, universities, overseas, etc. 


Figure 4 shows that over two fifths of the Department's research and development expenditure is incurred intramurally mainly at their research 
establishments, the remaining three fifths being work commissioned mainly from private industry. 


£4,407 million in 1991-92. In that period, 
Government expenditure on defence research 
and development reduced by 15.2 per cent 
whereas civil research and development 
reduced by 10.6 per cent. The defence share . 
of total Government research and 
development fell from 45.6 per cent in 
1987-88 to range between 44.2 and 44.9 per 
cent during the years 1988-89 to 1991-92 
whereas the civil share rose from 54.4 per 
cent in 1987-88 to fluctuate between 55.1 per 
cent and 55.8 per cent during the years 
1988-89 to 1991-92 (Figure 3). 


Over two fifths of the Department’s research 
and development expenditure is incurred 
intramurally mainly at their research 
establishments, the remaining three fifths 
being work commissioned mainly from 
private industry (Figure 4). The Department 
have one nuclear research establishment and _ 
five non-nuclear research establishments. 
Under the Government’s “Next Steps” 
initiative, a Defence Research Agency was 
set up on 1 April 1991 with the four main 
non-nuclear establishments forming the core 
of the Agency, which was established as a 
distinct organisation within the Civil Service, | 
operating ultimately as a Trading Fund 
accountable to the Secretary of State for 
Defence. 
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House of Lords Select Committee 
on Science and Technology 


In their 3rd Report of Session 1989-90, the 
House of Lords Select Committee on Science 
and Technology considered “Definitions of 
Research and Development”, subsequently 
referred to as the House of Lords Select 
Committee Report. The Committee 
concluded that: 


e the foremost area for improvement was 
the classification of defence research and 
development; 


e because of the large scale of defence 
research and development, 
misdescriptions of defence research and 
development unbalanced the total; 


e much of what was classed as defence 
research and development was not true 
research and development at all; 


e the practice of quoting combined defence 
and civil research and development 
expenditure should be discontinued 
because the two areas were distinct; 


e overstatement of Government funded 
research and development in the United 
Kingdom could distort international 
comparisons which were often made in 
Ministerial statements; and 


e the comparatively healthy position of the 
United Kingdom in the research and 
development international league owed 
much to the stated United Kingdom 
investment in defence research and 
development. 


The Committee recommended that the 
National Audit Office, with suitable technical 
support, should report on the Department's 
research and development expenditure, 
identifying how much of the expenditure fell 
within the Frascati definitions and within 
Accounting Standard 13 (revised). 


The Government’s response to the 
Committee’s report was published in March 
1991. It related the work which the 
Department had set in train to establish the 
possibility of producing more precise 
estimates of research and development under 
the Frascati definitions. On the practice of 
quoting combined defence and civil research 
and development expenditure, the response 
noted that some published sources already 
did so. However, the Government would 
welcome the separation in international 
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comparisons published by the Organisation 
for Economic Co-operation and Development. 


Scope of the National Audit Office 
examination 


In early 1990, the Department embarked 
upon a major review of the application of the 
Frascati definitions to defence research and 
development. In view of the current 
Departmental activity in this area, any 
detailed examination by the National Audit 
Office, as suggested by the House of Lords, 
would only duplicate the Department’s 
investigation and divert them from the task 
in hand. Moreover, the exercise proposed by 
the Select Committee would be outside the 
capability and resources available to the 
National Audit Office. 


The National Audit Office have undertaken 
an examination of the classification of 
defence research and development 
expenditure in order to add to the debate in 
a constructive way. In so doing the National 
Audit Office recognised that there were value 
for money issues involved in providing 
accurate information for Ministerial decision- 
making and to Parliament. The examination 
covered: 


(i) the applicability of the Frascati 
definitions and other related definitions 


of research and development, in 
particular their relevance to defence; 


(ii) the compilation of research and 
development expenditure statistics and 
the problems encountered in applying 
the Frascati definitions in the defence 
and civil sectors of the United Kingdom 
and overseas; 


(iii) the measures taken by the Department 
to implement the Frascati definitions 
and the effects of full implementation 
both on the Department and on 
Government statistics as a whole. 


In covering these themes, the National Audit 
Office have addressed the concerns of the 
Lords Select Committee (paragraph 1.16). 
During this study, the National Audit Office 
have sought the views of the defence trade 
associations, the Confederation of British 
Industry, the accountancy profession and 
specialist academics. A brief visit was also 
made to the headquarters of the Organisation 
for Economic Co-operation and Development 
in Paris. 
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Part 2: Frascati and related definitions of 
research and development 


This part of the Report considers: the 
applicability of Frascati definitions; the 
categorisation and definitions used by the 
Department; and the definitions contained in 
international and United Kingdom accounting 
standards. 


The Frascati Manual 


The Frascati definitions (Appendix) are the 
authoritative definitions of research and 
development expenditure which are accepted 
worldwide in the public sector and also in 
the private sector where they are reflected in 
accounting standards. Since the first edition 
of the ‘‘Frascati Manual” in 1963, there have 
been four revisions, the current version being 
the 1980 edition. The Manual has been 
written by and for the national experts in 
member countries of the Organisation for 
Economic Co-operation and Development. 
These experts collect and issue national 
research and development data and 
participate in the Organisation’s biennial 
research and development surveys. In 
general, the Manual recommends performer 
based reporting, but for analysis of 
Government intentions or objectives when 
committing money to research and 
development, the Manual specifies that 
reporting should be by the funder. 


The current revision of the Manual was 
launched by the Organisation for Economic 
Co-operation and Development in November 
1987 and is running six months to a year late. 
The Organisation expect a draft copy of the 
revised Manual to be in circulation by 1992 
but do not envisage publication before 1993. 


In the Department’s view, the Manual is not 
sufficiently defence-specific and the 
Organisation for Economic Co-operation and 
Development intend to make a special effort 
in the revised Manual to provide better 
guidance for measuring military research and 
development. The Organisation have 
produced a draft paper on defence research 
and development which discusses the 
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problems of the boundary between research 
and development and production. This paper 
in its final form will make recommendations 
for revision of the Manual in respect of 
defence aspects and will be circulated to the 
National Experts on Science and Technology 
Indicators who will decide upon the action 
required at a conference in Rome in 
September 1991. 


The House of Lords Select Committee Report 
made certain suggestions to the Organisation 
for Economic Co-operation and Development 
for improving the Manual: make it more user 
friendly; provide more detailed guidelines 
backed up by examples; record related 
activities which support research and 
development; define strategic research; 
consider the benefits of Accounting Standard 
13 as adapted from Frascati; and separate 
civil from defence research and development. 
The Central Statistical Office are liaising 
with the Organisation on this matter. 


Only research and development inputs are 
included in the body of the Manual. Two 
inputs are measured: expenditure on 
research and development, and people 
working in research and development. The 
Confederation of British Industry have made 
the point to the National Audit Office that 
research and development inputs can only be 
a proxy measure of competitive success in 
the absence of the measurement of research 
and development outputs. However, possible 
ways of measuring research and development 
output, such as innovations and patents are 
discussed in an annex to the Manual. 
Moreover, in July 1990 the Organisation for 
Economic Co-operation and Development 
produced a Technology Balance of Payments 
Manual and expect to issue an Innovation 
Manual during 1991. 


The Central Statistical Office have informed 
the National Audit Office that the 
Organisation for Economic Co-operation and 
Development are already giving priority to 
measuring the outputs from the research and 
development process and this is supported by 
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the Central Statistical Office. However, the 
Central Statistical Office take the view that if 
new data collection in this respect is 
proposed, care should be taken in limiting 
the burden on industry. 


The Department’s 
definition/categorisation of 
research and development 


For the purpose of the Statistical Supplement 
to the Chancellor’s Autumn Statement and 
the Cabinet Office’s Annual Review of 
Government Funded Research and 
Development, all Government departments 
are required to report research and 
development data based on the Frascati 
definitions. The Department maintain that 
the research and development figures which 
they supply are broadly in line with the 
Frascati definitions. 


No basic curiosity-driven research as defined 
by the Organisation for Economic Co- 
operation and Development is undertaken by 
the Department because all research is 
geared towards defence requirements. 
Therefore, in the Cabinet Office’s Annual 
Review of Research and Development, the 
Department categorise all of their research 
work as applied research which may be 
separated into strategic applied research and 
specific applied research. 


The Department’s Supply Estimate, on which 
the Annual Review data are based, assume 
that development continues to the point 
where production begins, since this 
corresponds to the usual contractual 
arrangements. Thus, the degree of novelty in 
such development work will vary. Also, there 
are difficulties in seeking to discriminate 
between different tasks where these are 
bound up in a single contract. The 
Department are currently examining the 
extent to which Frascati has been and can be 
applied to their research and development 
activities (paragraphs 4.9 to 4.19). 


The development work of the Department is 
related to specific equipment projects. To 
reduce and control financial and technical 
risk, the development programme is managed 
in three consecutive but distinct stages: 
feasibility study, project definition, and full 
scale development. These three phases for 
development conform with the pattern 


advocated in the Downey Report (1966) 
published in 1969 by the former Ministry of 
Technology. The philosophy underlying this 
report was the importance of investing time, 
money and effort at the early stages of a 
project to ensure that estimates of technical 
feasibility, timing and cost are as reliable as 
possible before proceeding to full 
development. The extent to which the 
Department’s three phases for development 
fall within the Frascati definition of 
experimental development is not clear but all 
three phases probably include some elements 
of Frascati and non-Frascati work. 


Definition/categorisation of 
research and development by the 
United States Department of 
Defense 


The United States spends more on defence 
research and development than any other 
member nation of the Organisation for 
Economic Co-operation and Development 
(paragraph 1.9). The stages of the research 
and development equipment cycle in the 
United States Department of Defense are 
research, exploratory development, advanced 
development, engineering development, and 
management and support. In connection with 
their inquiry, the House of Lords Select 
Committee asked all Government 
departments whether there would be any 
advantage in applying the American 
categories. The Department together with 
civil departments responded in the negative 
because the number of sub-divisions 
employed by the United States Department of 
Defense were seen to be potentially 
confusing and undesirable. 


Accounting standards for 
research and development 


A 1989 report “Defence Research and 
Development: A National Resource” by the 
Cabinet Office’s Advisory Council on Science 
and Technology recommended that the 
Department should publish research and 
development data according to the definitions 
of the former Accounting Standards 
Committee, now the Accounting Standards 
Board, in order to facilitate comparison with 
private sector information. The report was 
referring to the revised United Kingdom 
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Statement of Standard Accounting Practice 13 
where the definitions were based on those in 
the Frascati Manual. In principle, the 
Department support the report’s 
recommendation. 


The revised Accounting Standard 13 closely 
accords with International Accounting 
Standard 9 ‘“‘Accounting for Research and 
Development Activities”. Although the 
possibility of developing a European 
accounting standard was mentioned by the 
House of Lords Select Committee, the 
Institute of Chartered Accountants of 
Scotland has expressed the view to the 
National Audit Office that European 
accounting standards were unlikely to be 
developed as the current trend is towards 
harmonisation on a worldwide basis using 
statements such as International Accounting 
Standard 9. To date it seems that there has 
been no dialogue on the classification of 
research and development between the 
Organisation for Economic Co-operation and 
Development and the international 
accountancy profession. 
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Since January 1989, the revised United 
Kingdom Statement of Standard Accounting 
Practice 13 has also provided for the 
disclosure of research and development 
expenditure in company accounts. It mirrors 
the Frascati definitions in so far as it 
identifies the categories of pure (basic) 
research, applied, research and development, 
but it does not follow the detail precisely. 2.19 
The accountancy profession and the 

Confederation of British Industry have 

informed the National Audit Office that the 

impact of the revised Accounting Standard 13 

is only beginning to be seen but the defence 

industry has advised that the revised 

standard has in fact been implemented by 

the majority of member companies. 


A January 1991 report on “Accounting for 
Research and Development”, sponsored by 
the Board for Chartered Accountants in 
Business of the Institute of Chartered 


Accountants in England and Wales, 
concluded that the revised Accounting 
Standard 13 is limited in scope and provides 
inadequate guidelines to ensure consistent, 
comparable measurement of research and 
development expenditure. The report argues 
that the standard captures only a fraction of 
the activities and costs associated with 
product innovation and also provides 
inadequate guidelines on, for example, the 
nature of the costs to be included. But the 
Chartered Institute of Management 
Accountants has pointed out that, despite 
undeniable shortcomings in the revised 
standard, it is a great improvement on its 
predecessor and one very important factor, 
overlooked in the report, is the significance 
of the standard for gaining year-on-year trend 
information. The Accounting Standards 
Board are currently reviewing experience of 
the revised standard. 


The revised Accounting Standard 13 does not 
require separate figures for research and 
development expenditure to be identified in 
company accounts and the House of Lords 
Select Committee have suggested that this 
should be done. However, according to the 
Chartered Institute of Management 
Accountants the commercial and market 
sensitivity of the information was one reason 
why the revised Accounting Standard 13 
adopted a more cautious approach to 
disclosing these separate figures. 


The National Audit Office obtained the views 
of the accountancy profession on whether 
there should be one set of research and 
development definitions for both the public 
and private sectors. Generally, it was agreed 
that there should be uniformity as this would 
bring the benefits of familiarisation and 
standardisation and ease comparison between 
the two sectors. But standard definitions 
would need to be supported by guidance 
notes on their application which would cover 
the various economic sectors and provide 
illustrative examples in real life settings —a 
point also made by the Confederation of 
British Industry. 
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Part 3: Compilation of research and 
development expenditure statistics 
and associated problems 


This part of the Report examines how 
research and development expenditure 
statistics are compiled and the problems 
experienced in applying the Frascati 
definitions internationally and in the 
Government and industrial sectors of the 
United Kingdom. 


Compilation of international 
research and development 
statistics 


The Organisation for Economic Co-operation 
and Development depend upon member 
Governments, in responding to the 
Organisation’s Biennial Benchmark 
International Surveys, to supply information 
from national surveys and budgets. These 
surveys are carried out by sending 
questionnaires to the relevant authorities 
responsible for statistics in member 
countries. They are updated by a twice 
yearly questionnaire. The Organisation also 
extract data from national publications, 
especially where there is an annual national 
survey of research and development. 


Most major countries carry out a full annual 
survey (or census) of industry but some 
undertake full surveys biennially like the 
Organisation itself. For example, France and 
Japan carry out an industry census annually 
and the United States and West Germany 
undertake one biennially. Every major 
country except the United Kingdom was able 
to provide the detailed figures required for 
the Organisation’s 1990 biennial survey. 


The United Kingdom was unable to supply 
all the necessary figures because it relies on a 
full survey of industry every four years, 
updated annually by sample surveys 
(paragraph 3.8). The Central Statistical Office 
of the United Kingdom have now supplied 
the data required following publication in 
July 1991 of the final results of the 1989 


quadrennial survey. In the view of the 
Central Statistical Office, a full annual census 
would be out of proportion to the policy 
needs for monitoring and an undue burden 
on industry. 


Although the Organisation for Economic Co- 
operation and Development consider that all 
member countries generally collect and 
report research and development in line with 
the Frascati Manual, they accept that some 
detailed national specifications may vary 
from the Organisation’s definitions. However, 
the Organisation consider that differences are 
generally likely to be too small to affect the 
indicators quoted in publications. 
Nonetheless, in making international 
comparisons, caution must be exercised 
because of the differences in the way which 
the information may have been collected and 
the extent to which the Frascati definitions 
may or may not have been applied. Factors 
common to most countries are: 


e the exclusion of small firms; 


e differences in reporting years between 
Government and industry; 


e a difference, to a greater or lesser extent, 
between what Government reports as 
having paid to industry and what 
industry claims to have incurred in 
undertaking Government funded research 
and development. 


Although most member countries exclude 
small firms from their industry census, the 
classification of a small firm varies from that 
in the United Kingdom. Other major member 
nations most often define a small firm as a 
business with less than 50 employees 
whereas in the United Kingdom a small firm 
is designated as having less than 200 
employees. The exclusion of small firms from 
the Central Statistical Office’s four-yearly 
benchmark surveys is a direct result of 
Government policy to limit the form-filling 
burden on small companies. The Central 


15 


CLASSIFICATION OF DEFENCE RESEARCH AND DEVELOPMENT EXPENDITURE 


Boil 


3.8 


3.9 


Statistical Office have estimated that the 
spending on research and development by 
such firms was only about £212 million in 
1989 and they do not consider that a full- 
scale sample survey of these small firms 
would be justified. 


Compilation of research and 3.10 


development statistics in the 
United Kingdom 


The Central Statistical Office compile the 
statistics on research and development 
expenditure in the Cabinet Office Annual 
Review of Government Funded Research and 
Development from two main sources: 
Government departments and industry. 


Data on Government funded research and 
development are collected by carrying out a 
full survey each year of all Government 
departments and other central Government 
bodies which carry out or directly 
commission research and development. For 
research and development performed in 
industry full surveys of expenditure are 
undertaken every fourth year to provide a 
benchmark, and sample surveys (covering 
about 75 per cent of reported research and 
development expenditure in a benchmark 
year) are carried out in the intervening years. 
The last full quadrennial benchmark survey 
of industry was for the calendar year 1989 
and final results were published in July 1991. 
For the first time the 1989 industry survey 
includes: 


e a split between civil and defence research 
and development; 


e identification of the sources for industrial 
defence research and development: 
Government, overseas and industry’s own 
funds; 


e a published estimate for small firms 
included in the total of industrial 
research and development expenditure. 


The national gross expenditure on 
research and development 


The national gross domestic expenditure on 
research and development from all sources is 
derived from the data in the performer- 
related industry surveys and by extracting 
the intramural elements from Government 


surveys. This method is used by other 
member countries of the Organisation for 
Economic Co-operation and Development. 
The United Kingdom’s gross expenditure on 
research and development from all sources, 
both public and private, was £11.5 billion for 
1989 (Table 1). 


One of the conclusions in the House of Lords 
Select Committee Report was that 
overstatement of the Department’s research 
and development expenditure could result in 
an appreciable distortion of the United 
Kingdom’s position in the context of 
international comparisons. For Government 
funded research and development, the 
Department are responsible for reporting both 
intramural and extramural research and 
development expenditure. However, the only 
figure reported by the Department for the 
purpose of calculating the nation’s gross 
expenditure on research and development 
from both public and private sources is the 
intramural research and development 
undertaken by the Department as a 
performer. This amounted to £939 million in 
1989 ie less than one twelfth of the £11.5 
billion total. 


The problems which the Department have 
experienced in defining research and 
development relate mainly to their 
extramural research and development 
(paragraph 3.14); and for the purpose of the 
national gross expenditure on defence 
research and development, that is reported 
by industry as the performer and not by the 
Department. The National Audit Office note, 
therefore, that the Department’s input should 
not have unduly affected the total. 


Problems in the Government 
sector 


(a) Defence 


For Government funded research and 
development, the Department, as the funder, 
are required to report on intramural and 
extramural expenditure; and for the purpose 
of the national gross expenditure on research 
and development, they are required, as the 
performer, to report on intramural 
expenditure alone. The figures reported by 
the Department are drawn from the vote 
accounting system but this information is not 
sufficiently disaggregated to allow further 
analysis of Frascati and non-Frascati 
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elements. The main problems are that the 
vote accounting structure does not provide a 
framework for the fine details needed to 
record Frascati research and development 
expenditure; and information on Frascati 
research and development is not always 
available at levels below the main vote 
structure. 


The Department have experienced particular 
problems applying the Frascati definitions to 
experimental development and the bulk of 
any potential misclassification is likely to 
occur at the development/production 
interface, a definitional ‘grey’ area. 

Examples of expenditure which are counted 
as research and development under the 
present system but which may not fall within 
the Frascati definitions include some 
feasibility studies (other than those on a 
research project), final product or design 
engineering (including some post design 
services), production related technical 
demonstrations and copies of prototypes after 
the successful testing of the original. 


Most of the Department’s intramural research 
and development occurs within their 
research establishments and the Department 
have concluded that generally sufficient 
information is held to enable estimates of 
expenditure falling within the Frascati 
definitions to be improved. The difficulties 
faced by the Department in accurately 
estimating Frascati research and development 
expenditure relate mainly to extramural 
research and development of which the 
major part is development. Non-Frascati 
activities such as production engineering and 
pre-production tooling are frequently 
included in development contracts. To 
identify the Frascati elements, detailed 
information would be needed of how the 
various contract payments are spent but the 
Department do not have this information 
(paragraph 4.7(iii)). Also, the Department 
consider that even within defence 
contractors’ own organisations, such 
information may not be readily available 
particularly where work is sub-contracted. 


(b) Civil 


Generally, other Government departments do 
not experience the problems the Department 
have in extracting Frascati from the vote 
accounting system because their research and 
development fall more naturally within the 
Frascati boundary. For defence equipment, 
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production follows on from development as a 
rule rather than as an exception. However, 
the research and development of civil 
departments rarely lead to Government 
funded production and therefore definitional 
problems on the development/production 
border do not occur. From the evidence 
presented to the House of Lords Select 
Committee, Research Councils encountered 
most problems on the borderline between 
basic and applied research and civil 
departments have difficulty with the 
interpretation of applied research. 


Problems in the industrial sector 


The Confederation of British Industry have 
informed the National Audit Office that few 
companies appear to experience any 
difficulty differentiating between basic and 
applied research as defined by Frascati 
because almost all industrial research is 
applied. Similarly, most companies appear to 
have no problem in differentiating between 
research and development. But by far the 
greatest problem companies experience with 
Frascati is differentiating between 
‘experimental development’ and subsequent 
product and process development. 


In relation to the Central Statistical Office’s 
industry surveys: 


e The Society of British Aerospace 
Companies have advised the National 
Audit Office that some member 
companies report defence research and 
development contract work in accordance 
with the Department’s traditional 
definition of research and development 
(ie according to the Downey 
procedures — paragraph 2.11) whereas 
others report defence research and 
development expenditure to accord with 
the Frascati definitions. The Society view 
the Frascati definition of experimental 
development as covering: all of the first 
two stages of the Department’s 
development programme ie feasibility 
study and project definition; and a small 
part of the third stage, full development, 
relating primarily to demonstration. 


e British Aerospace, in giving evidence to 
the House of Lords in 1989, stated that 
they have not in the past identified 
research and development according to 
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the Frascati definitions but action was in 
hand to do so. 


e The National Audit Office were advised 
by the Defence Manufacturers 
Association that there was little to 
indicate any general discrepancy in 
allocation between the Department and 
industry; member companies had not 
found any particular need to consider the 
distinction between the Department’s 
traditional definition of research and 
development and that of Frascati. The 
Association saw the Frascati definition of 
experimental development as being 
capable of being subdivided in practice 
into feasibility study, project definition, 
full development and post design 
services. 


3.18 Not all research and development performed 


by industry for defence purposes is funded 
by the Department. For example, there is 
industry’s own private venture research and 
development often connected with the 
development of exports for defence 
equipment. For the first time, the industry 
survey for 1989 has identified the extent of 
private investment in defence research and 
development. The results of the survey show 
that industry itself funded £426 million ie 26 
per cent of the total of £1,653 million defence 
work performed by industry in 1989. The 
Department have doubts about this seemingly 
high figure. 


The discrepancy between 
Government and industry surveys 


When the industry figures of research and 
development performed in-house but funded 
by Government are compared with the 
Government figures of research and 
development expenditure paid to industry, 
there are certain apparent differences. For 
example, in 1989-90 Government 
departments reported that £1,582 million 
research and development had been funded 
to industry but industry reported only £1,249 
million as having been spent on Government 
work in 1989: a discrepancy of £333 million. 
A discrepancy of about £500 million had 
existed since the earlier 1985 benchmark 
industry survey. Therefore, the most recent 
benchmark survey for 1989 has resulted in 
the discrepancy being reduced by about £170 
million. 
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Since the Department fund over three 
quarters of all Government research and 
development payments to industry, much of 
this discrepancy will relate to defence. For 
1985, the defence industry reported that 
£1,082 million had been spent on work 
funded by the Department but the 
Department reported that for 1985-86 £1,317 
million (£1,488 million less VAT) had been 
funded to industry, a difference of £235 
million. For 1989-90 the Department have 
reported that £1,156 million was paid to 
industry whereas the defence industry have 
reported that only £943 million was incurred 
on Government work in 1989, a difference of 
£213 million. Thus, the defence discrepancy 
has reduced by some £20 million between 
the two benchmark survey years. 


The 1990 Cabinet Office Annual Review gave 
the following reasons for the discrepancy: 


e the Government funding figures include 
the profit element of any research and 
development contract placed with 
industry whereas the industry figures 
exclude profit; 


@ acompany sub-contracting from another 
company may not recognise the 
Government as the ultimate source of 
funds; 


@ acompany sub-contracting from another 
company may not appreciate that the 
work it is carrying out is an essential 
element of the contracting company’s 
research and development programme 
and may not therefore classify it as 
research and development in the industry 
survey. 


In January 1991, the Department undertook 
some initial exploratory work on the 
discrepancy which suggested that the small 
firms and profit exclusions from the 
Government figures were the main factors 
together with misattribution by either or both 
the Department and industry. The 
Department have set up a working group 
with the Central Statistical Office and the 
defence industry to investigate the 
discrepancy. The action being taken is 
considered further at paragraphs 4.9 and 4.13 
to 4.19. 
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An industry view of the 
Department’s research and 
development expenditure 


In 1987, the Society of British Aerospace 
Companies produced a paper which 
suggested that just over 50 per cent of the 
Department’s research and development 
expenditure supported work which was truly 


within the spirit of the Frascati definition. In 
considering the nature of the work funded by 
development contracts for a range of 
aerospace products, the Society included all 
work up to project definition stage but 
excluded most of the full development. In 
the absence of any evidence to the contrary, 
all intramural research and development 
work and all extramural research work was 
assumed to be truly Frascati. 
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4.2 


4.3 


Part 4: Action by the Department to 
implement the Frascati definitions 
and the implications of full 


implementation 


The Department, as the funder, must report 
on intramural and extramural research and 
development expenditure for the calculation 
of the United Kingdom’s total Government 
funded research and development; and on 
intramural expenditure alone, as the 
performer, for calculation of the nation’s 
gross expenditure on research and 
development. As Part 3 shows, since the mid 
1980s there has been a growing recognition 
within the Department that published 
estimates of defence research and 
development might be capable of improved 
alignment with the Frascati definitions. This 
part of the Report examines the various 
measures taken by the Department to 
implement the Frascati definitions. It also 
considers the implications if the Department 
were to fully implement them. 


Action taken by the Department 


The 1986 pilot study 


In 1986, the Department undertook a pilot 
study to obtain a very broad indication of 
how much defence intramural and 
extramural research and development 
expenditure reported by the Department 
might be non-Frascati. The study tentatively 
concluded that around 25 per cent of the 
expenditure involved in the sample might 
not be pure Frastati. However, for a more 
accurate result, clearer guidance notes would 
need to be produced with a detailed 
interpretation of the grey definitional areas in 
the Frascati Manual. 


The 1987 survey 


The Department instigated a follow-up 
survey in 1987. This involved issuing a 
questionnaire on a sample of forty eight 
projects. Individual project offices were asked 
to provide reported extramural expenditure 
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for 1985-86 broken down into Frascati and 
non-Frascati categories. The survey also 
covered intramural research and 
development expenditure at the 
Department’s research establishments which 
was analysed in its entirety. After completion 
of the main enquiry, follow-up audits were 
conducted on ten of the projects to provide 
an assessment of the quality of returns and to 
uncover areas of difficulty (paragraph 4.5). 


The survey report produced in February 1988 
identified weaknesses in the Frascati 
definitions. Subject to these, and to a shortfall 
in the availability of project data, the survey 
suggested that 79 per cent of the sample’s 
reported research and development 
expenditure in 1985-86 was true Frascati. 
The Department’s intramural expenditure at 
research establishments, which concentrates 
on research and on the more exploratory 
stages of development, was found to conform 
fairly closely with Frascati, only 9 per cent of 
the sample expenditure being discounted as 
non-Frascati. On the other hand, 73 per cent 
of the sample’s extramural research and 
development expenditure, contracted out by 
the three Systems Controllerates, met the 
Frascati definitions. 


The results of the survey were subject to 
qualification because of the following factors: 


e Sampling error. The randomly selected 
sample might or might not be typical of 
the population as a whole. The survey’s 
analysis showed that 79 per cent was the 
most probable proportion of measurable 
Frascati research and development for 
1985-86 but this proportion might have 
been as low as 71 per cent or as high as 
87 per cent depending upon the projects 
selected; 


e Bias in questionnaire completion. In 
general, respondents to the project 
questionnaire found that it was difficult 
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to determine when work was no longer 
novel. Specific problems were 
encountered in deciding the correct 
treatment for VAT, product marketing, 
feasibility studies and post-design 
services. Moreover, the follow-up audits 
of ten projects highlighted two basic 
problems. First, in the face of definitional 
problems, respondents often made hasty 
interpretations of Frascati. Second, project 
data needed for completion of the 
questionnaire-were often not available 
within the Department but were held at 
contractors; and 


e Omission. The survey did not examine 
areas currently counted as production 
which might contain elements of research 
and development. 


The Department now consider that there is 
no certainty about the over-recording of 
research and development and the survey 
simply shows the potential for misattribution. 
They intend to undertake a further survey in 
collaboration with the defence industry to 
afford greater access to project information 
than had been available for the 1987 survey 
(see paragraphs 4.18 and 4.19). 


The 1989 review of research and 
development expenditure estimates 


In February 1989, the Department’s Statistics 
Division was tasked with producing the first 
stage of a study on how the Department’s 
research and development expenditure 
estimates might be brought into line with the 
Frascati definitions. A report was produced in 
August 1989. The main findings were: 


(i) In general. The Department’s present 
method for estimating was based on the 
vote accounting system and might be 
significantly overstating the total 
expenditure which should be allocated 
to research and development under the 
Frascati definitions. Although there 
were areas where the use of the 
Frascati definitions would increase the 
level of the research and development 
estimates, these would be more than 
offset by areas where estimates would 
be reduced. 


(ii) Intramural research and development. 
Information existed within the 
Department, principally at research 
establishments, in an accessible form 
which would enable the Frascati 
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guidelines to be more closely applied. 
The mechanisms set up to provide 
routine estimates would need to be 
carried forward into the Defence 
Research Agency for the four 
establishments involved. 


(iii) Extramural research and development. 
Much of the information required to 
differentiate properly between Frascati 
and non-Frascati research and 
development resided with defence 
contractors and in a form that 
prevented easy extraction. Moreover, it 
was unlikely that all contractors would 
be willing to provide the information on 
either a voluntary or mandatory basis 
until such time as their internal 
accounting systems could generate the 
required Frascati breakdown. 


(iv) Short-term prospect. It should be 
possible to move intramural research 
and development estimates on to a full 
Frascati basis but only limited progress 
could be made towards refining the 
extramural research and development 
estimates. 


(v) Long-term prospect. Whether Frascati 
research and development estimates 
could be produced accurately rested 
critically on whether the necessary 
information could be obtained from 
defence contractors. The supply of such 
information would probably have to be 
made a condition of contract before the 
required accuracy could be achieved. 
Even if objections from contractors 
could be overcome, the introduction of 
contract conditions for the supply of 
data could only be imposed on new 
contracts; and it would take many years 
before all extramural research and 
development expenditure was covered 
by such provisions. 


(vi) Vote structure. Full implementation of 
Frascati would require a refinement of 
the vote structure. 


In February 1990, the Department agreed to 
implement the report’s recommendations as 
follows: 


In the short term 


(i) Intramural research and development 
estimates should be refined by 
exploiting the information held in the 
research establishments; and steps 
should be taken to ensure that such 


nn EE 


21 


CLASSIFICATION OF DEFENCE RESEARCH AND DEVELOPMENT EXPENDITURE 


4.9 


information would continue to be 
available after formation of the Defence 
Research Agency. 


(ii) Extramural research and development 
estimates should also be refined by 
another more comprehensive survey of 
projects, thereby fully utilising 
whatever information was already 
available in the Department. From the 
survey results, a statistical estimate of 
Frascati extramural expenditure should 
be produced. 


In the longer term 


(iii) Further consideration should be given 
to the introduction of contract 
conditions to require contractors to 
supply the necessary information. 


(iv) Aggregate information should be 
obtained from the Central Statistical 
Office’s industry surveys to help refine 
the Department’s own estimates. 


(v) There was a need to consider revising 
the vote accounting system. 


Current departmental action on 
implementation 


Following the decisions on implementation, 
in March 1990 the Department drew up an 
Implementation Work Plan. This included 
the provision of new refined guidance notes 
by June 1990 and the production of refined 
intramural and extramural estimates of 
Frascati research and development 
expenditure by April 1991. The organisation 
of a working group composed of the 
Department, the Central Statistical Office and 
industry was also part of the work plan. This 
work took on a new level of priority 
following the House of Lords Select 


Committee’s Report published in March 1990. 


Progress achieved is summarised in the 
following paragraphs 


(a) New guidance notes 


The revised guidance notes were produced 
by June 1990. These were a revision of the 
earlier 1988 notes, rewritten in a form to 


make the task of applying the Frascati 
definitions less onerous. 
(b) Intramural research and development 


In early July 1990 a questionnaire was 
produced and issued to all of the research 
establishments. This was more 
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comprehensive than the version used during 
the 1987 survey. 


Work has proceeded well and on schedule. 
Most research establishment staff found it 
possible to determine fairly clearly which 
activities fell within the Frascati definitions 
and, hence, the proportions of their total 
budgets which were attributable to Frascati- 
defined research and development. 
Preliminary results as at August 1991 
indicated that roughly 15 per cent of 
currently published estimates of intramural 
research and development expenditure for 
1988-89 would not be counted as research 
and development under a stricter application 
of the Frascati definitions. However, the 
figure of 15 per cent may change when 
queries on several questionnaires are 
resolved and may not apply to years other 
than 1988-89. The Department intend to 
investigate further. 


(c) Extramural research and development 


The Department planned to tackle the 
problems of extramural research and 
development estimation in three ways: 
exploitation of the knowledge of 
Departmental project staff about the work 
involved in executing research and 
development contracts, use of the Central 
Statistical Office industrial survey of research 
and development, and a joint 
Departmental /Central Statistical Office 
investigation with the defence contracting 
industry. 


(i) Exploitation of internal departmental 
knowledge 


The Department considered that they needed 
to undertake a more comprehensive exercise 
than the 1987 survey over a longer timescale 
to enable a view to be taken on the level of 
over or under recording of extramural 
research and development expenditure across 
the whole Department. A pilot study was 
mounted in June 1990 with five projects 
targeted. The early results from this exercise 
have not been encouraging due mainly to the 
small size of the sample and at least two of 
the contracts having no substantial 
expenditure for 1988-89, the period of the 
survey. 


Each of the three remaining projects involved 
significant amounts of development work in 
addition to elements that were clearly not 
innovative. Although project staff understood 
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the basis of the Frascati classifications, all felt 
unable to judge the degree to which the work 
done by the companies concerned was truly 
innovative. Moreover, where projects had 
been let on a competitive tender there was 
no way in which project staff could 
distinguish between costs and profits. 


The Department, therefore, considered that 
knowledge held internally was insufficient to 
improve the extramural research and 
development estimates. At best, such 
knowledge was patchy and could not form a 
basis for better statistical estimation. 
Consequently, a more detailed survey is not 
being pursued. 


(ii) Use of the 1989 industry survey 


The Department have had discussions with 
the Central Statistical Office about the 
potential use of their 1989 survey of 
industrial research and development. 
Although it is possible to derive estimates of 
the costs incurred by industry in carrying out 
Frascati-defined research and development 
under defence contracts, the accuracy and 
completeness of these estimates would 
require further investigation. 


(iii) Tri-partite working group 


The House of Lords Select Committee’s 
Report highlighted the discrepancy on 
extramural research and development 
between the Department’s reported funding 
to industry and the sum reported by industry 
as having been incurred on Government 
work. The Department believe that the best 
way to tackle the problem is to consult with 
the Central Statistical Office and the defence 
industry in order to try and establish where 
differences in interpretation or application of 
the Frascati definitions occur. As stated 
above a working group has been set up 
which is looking specifically at the 
Department’s estimates of extramural 
research and development and how they 
relate to the corresponding industry 
estimates. The intention is to follow a 
suitable sample of contracts through the 
examination processes that are carried out 
within the Department, in industry and in 
the Central Statistical Office to arrive at the 
various published estimates. The aim is to 
identify the extent and composition of any 
differences. 


4.19 


The defence industry is represented in the 
working group by firms such as British 
Aerospace, GEC, Rolls Royce and Vickers 
Defence Systems. Furthermore, the following 
associations have been consulted and 
encouraged to nominate representatives from 
member companies: the Confederation of 
British Industry, the Society of British 
Aerospace Companies, the Electrical 
Engineering Association, the Defence 
Manufacturers Association and the Defence 
Industries Council. The working group 
convened in May 1991 and plans to produce 
its first interim report by the end of the year. 


Implications of the full 
implementation of Frascati 


The potential benefits 


Much of the point of a centralised survey of 
research and development expenditure, such 
as in the Cabinet Office Annual Review, is 
lost if figures from different sources are 
calculated on different bases. What is needed 
is uniform definition of research and 
development and a consistent basis for 
assessment of expenditure. 


In the view of the National Audit Office a 
number of benefits would ensue from a more 
complete employment of Frascati by the 
Department: 


(i) Best accounting practice. The 
Department would be falling in line 
with best accounting practice as 
embodied in Accounting Standard 13 
(Revised) and as recommended by the 
Advisory Council on Science and 
Technology (paragraph 2.13). The extent 
to which industry have complied with 
Accounting Standard 13 (Revised) is not 
yet clear (paragraph 2.16). 


(ii) Uniformity of reporting. The 
Department would be reporting 
research and development expenditure 
on the same basis as the Government 
civil sector and industry. However, in 
the Central Statistical Office’s industry 
survey not all of the defence industry 
are reporting research and development 
expenditure according to the Frascati 
definitions (paragraph 3.17). 


(iii) More informed decision making. More 
accurate information would be given to 
central Government as the basis for 
policy formulation and the 
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determination of research and 
development priorities throughout the 
Government sector. For example, since 
1987-88 there has been a Government 
policy to achieve a gradual reduction in 
defence research and development to 
free resources for civil purposes 
(paragraphs 1.13 and 1.14). If estimates 
of defence research and development 
are misaligned, doubt must be cast over 
the ability of Government to implement 
this policy effectively. 


(iv) More realistic comparisons. In theory, a 
better basis would be provided for 
comparisons between research and 
development investment in the United 
Kingdom defence and civil sectors and 
between the United Kingdom and other 
countries. However, in practice other 
nations might not be measuring up to 
the Frascati standards. 


Effect on research and development 
expenditure statistics 


The results of the Department’s 1987 survey 
must be treated with some caution 
(paragraphs 4.3 to 4.6). Nonetheless, the 
National Audit Office consider that these 
results are sufficiently robust to extrapolate 
to give an indication of the extent to which 
the Department’s overstatement of research 
and development expenditure may be 
affecting the published statistics. The 
preliminary results of the Department’s 
current survey update the position on 
intramural research and development 
expenditure (paragraphs 4.11 and 4.12). 


(a) Government funded research and 
development 


These figures are reported in the Cabinet 
Office’s Annual Review of Government 
Funded Research and Development. The 
1987 survey concluded on a sample basis that 
about 79 per cent of the relevant defence 
research and development expenditure might 
be Frascati (paragraph 4.4). For 1989-90, this 
would mean that Government funded 
defence research and development 
expenditure amounted to about £1.7 billion 
and not the reported £2.2 billion (Figure 3). It 
would comprise 39 per cent of total 
Government research and development 
expenditure instead of 45 per cent. If the 
most recent results of the Department’s 
survey of intramural research and 
development were substituted for the 
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intramural indications in the 1987 survey, 
these figures would not change appreciably 
(paragraph 4.12). 


(b) National gross expenditure on research 
and development 


These figures, which are also reported in the 
Cabinet Office Annual Review, are used to 
show total United Kingdom expenditure on 
research and development funded from all 
sources, both public and private. For this 
purpose, the Department’s only input is on 
intramural reseach and development, 
industry being responsible for reporting on 
extramural research and development. The 
Department reported £939 million intramural 
research and development expenditure for 
1989-90. According to the 1987 survey 
(paragraph 4.4), this figure may be overstated 
by 9 per cent which would amount to £85 
million for 1989-90. Thus, the United 
Kingdom’s gross expenditure on research and 
development in 1989 of £11,532 million 
(Table 1) would be overstated by £85 million. 
This would not be sufficient to affect the 2.3 
percentage of gross domestic product 

(Figure 1). 


If the more recent preliminary results from 
the Department’s current survey of 
intramural research and development are 
applied (paragraph 4.12), the Department’s 
£939 million intramural expenditure would 
be overstated by 15 per cent or £141 million. 
This would still not be sufficient to affect 
materially the nation’s gross expenditure on 
research and development as a percentage of 
gross domestic product. 


(c) International comparisons 


Taking the intramural results from the 
Department’s current survey and the 
extramural results from the 1987 survey, the 
United Kingdom would still remain one of 
the three big defence spending nations of the 
Organisation for Economic Co-operation and 
Development with about two fifths of its 
Government funding for 1989 devoted to 
defence instead of almost a half. For national 
gross expenditure on research and 
development as a percentage of gross 
domestic product, the United Kingdom’s 
position internationally would be unaltered 
(Figure 1). Total Government funded research 
and development as a percentage of gross 
domestic product would also remain 
unchanged at 0.9 per cent for 1989 with the 
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United Kingdom maintaining its ranking 
amongst the eight nations in Figure 2. 


Effect on the Department 


The Department consider that there is no 
domestic benefit to them in fully 
implementing the Frascati definitions: 


e any exercise to implement Frascati fully 
would be a drain on resources but the 
National Audit Office note that the 
Department have not calculated the 
extent of this; 


e for vote accounting and normal 
managerial and organisational purposes, 
the Department do not need to know how 
much they are spending on research and 
development which meet the Frascati 
definitions; and 


e the objective of the Department’s research 
and development programme is to meet 
the equipment needs in a timely and cost 
effective manner and implementation of 
Frascati would not aid this objective but 
could detract from it in terms of time and 
cost. 


The Department’s current exercise of 
attempting to produce a central estimate of 
Frascati intramural and extramural 
expenditure from information held within 
the Department has not been costed but it 
will involve the resources of Statistics 
Division, research establishments and project 
offices. From the viewpoint of the Central 
Statistical Office, it is not necessary to have 
auditable vote-based figures. Nonetheless, 
after completion of the current exercise, the 
Department will consider the longer term 
prospect of incorporating Frascati research 
and development estimates into the Supply 
Estimate and the vote accounting structure 
(paragraphs 4.7 and 4.8). However, the 
Department believe that they must take into 
account the size, complexity and cost of such 
a task and attempt to minimise the disruption 
to important departmental processes by 
careful and properly phased implementation 
of any changes. The National Audit Office 
note that, in connection with the 
Department’s New Management Strategy, 
modifications to the vote structure are 
intended by 1 April 1993. 


Currently, Frascati intramural expenditure at 
the research establishments is fairly 
accessible. However, with the formation of 
the Defence Research Agency in the Spring 
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of 1991 such information may not be so 
readily available. The Agency will be 
introducing a new common internal 
accounting system for the four 
establishments involved and there will be no 
guarantee that Frascati research and 
development can still be extracted. 
Furthermore, the Department’s relationship 
will be on more of a contractual basis with 
less opportunity for the Department to 
scrutinise the Agency’s internal expenditure. 
Nonetheless, the Department could exercise 
their rights as the owner of the Defence 
Research Agency to have the necessary 
information supplied to them. 


The greatest problem which the Department 
face both in size and definition relates to 
extramural research and development. The 
Department do not have the necessary 
detailed information and it is by no means 
certain that defence contractors either have 
that data now or will have it in the future. 
Accounting Standard 13 (Revised) requires 
companies to account for their own self- 
funded research and development 
expenditure but they are not required to 
account for work commissioned by others 
from them. 


Effect on the defence industry 


If the Department find that they are unable to 
complete extramural Frascati estimates from 
their own records, they intend to approach 
defence contractors to see what information 
might be supplied on a voluntary basis. 
However, in the longer term, a contract 
condition might have to be introduced for the 
supply of the required information 
(paragraph 4.7(v)). The Department consider 
that the difficulty and cost of implementing 
and monitoring such a contract condition 
would be substantial because they would be 
asking for information which was not readily 
available within the contractor’s own 
organisation, particularly where sub- 
contractors were employed, and thus the 
costs incurred would be passed on to the 
Department. In addition, lengthy negotiation 
would be required to establish this practice 
which would represent a reversal of the 
Department’s “hands off” approach to 
contractors by requiring closer scrutiny of 
their operations and could substantially 
jeopardise the procurement policy of 
competitive tendering which requires no 
background information on pricing. 
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4.32 The Department accept the need for co- 


operation with industry rather than 
compulsion. Any attempted compulsion by 
means of a special contract condition would 
have cost implications for the Department’s 
procurement budget and might influence the 
placing of competitive contracts. Moreover, 
the Department would be unable to impose a 
contract condition for the supply of Frascati 
information on existing contracts. 


4.33 Some parts of the defence industry told the 


National Audit Office that they did not 
consider full implementation of Frascati by 
the Department would create any special 
problems for them and that it would result in 
a more accurate breakdown between 
investment in “‘true research and 
development” and major development work. 
However, others take the view that further 
refinement is unlikely to result in tangible 
benefits with any change only increasing the 
administrative burden on companies. 
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Appendix 1 
The Frascati definitions 


1. The Frascati Manual gives the basic definition of research and development as 
follows: 


“Research and experimental development comprise creative work 
undertaken on a systematic basis in order to increase the stock of 
knowledge, including knowledge of man, culture and society and the use 
of this stock of knowledge to devise new applications.” 


2. Research and development is then further defined under three activities: basic 
research, applied research and experimental development. 


(i) Basic research is experimental or theoretical work undertaken primarily to 
acquire new knowledge of the underlying foundation of phenomena and 
observable facts, without any particular application or use in view. 


(ii) Applied research is also original investigation undertaken in order to acquire 
new knowledge. It is, however, directed primarily towards a specific practical 
aim or objective. 


(iii) Experimental development is systematic work, drawing on existing knowledge 
gained from research and/or practical experience that is directed to producing 
new materials, products or devices, to installing new processes, systems and 
services, or to improving substantially those already produced or installed. 


3. The Manual discusses the boundaries of research and development highlighting 
that the basic criterion for distinguishing research and development from related 
activities is the presence in research and development of an appreciable element of 
novelty. Certain activities are specifically excluded from the measurement of 
research and development: 


(i) Education and training 


(ii) Scientific and technological innovation — new product marketing, patent and 
licence work (the legal and administrative part), financial and organisation 
changes, final product or design engineering, tooling and industrial engineering 
and manufacturing start up. 


(iii) Other related scientific and technical activities — scientific and technical 
information services, general purpose data collection, testing and 
standardisation, feasibility studies (other than on research projects), specialised 
medical care and policy related studies. 


(iv) Other industrial activities — prototypes which are copies of the original, pilot 
plants once operating commercially, trial production and trouble-shooting. 


4. The Manual gives the following definition of defence research and development: 


‘Defence includes all research and development programmes undertaken 
primarily for military reasons regardless of their content or whether they 
have secondary civil applications. It includes nuclear and space research 
and development undertaken for military purposes. It does not include 
civil research and development financed by ministries of defence, for 
instance, on meteorology or telecommunications.” 


Guidance on how this definition for defence is to be applied is not provided in the 
Manual. 
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Reports by the Comptroller and Auditor 
General 
Session 1991-92 


The Comptroller and Auditor General has to date, in Session 1991-92, 
presented to the House of Commons the following reports under Section 9 of 
the National Audit Act, 1983: 
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